18.05 Lecture 30 
April 27, 2005 



Bayes Decision Rule 

^(l)ai((5) + ^{2)a2{d) minimize. 



S = {Hi: ^ > It^; : if <;Hi or H2 : if =} 

/2(x) ^(1) 



Example: see pg. 469, Problem 3 
Ho : h{x) = 1 for < a; < 1 
Hi : f2ix) = 2a; for < a; < 1 
Sample 1 point xi 
Minimize 3ao{5) + lai{6) 



Simplify the expression; 



^ = {Ho ■ W— > I'^Hi : ^ <l; either if equal} 
2xi 3 2xi 3 



3 3 
5 = {Ho:xi< -iHi ■xi> -} 



Since a;i is always between and 1, Hq is always chosen. S = Hq always. 
Errors; 

ao{S) = Po(<5 ^ i?o) = 

We made the ao very important in the weighting, so it ended up being 0. 

Most powerful test for two simple hypotheses. 

Consider a class = {S such that ai{S) < a e [0, 1]} 
Take the following decision rule; 

/2(x) /2(x) 
Calculate the constant from the confidence level a: 

o^iiS) = Pi(<5 7^ H{) = Pi(^44 < c) = a 

Sometimes it is difficult to find c, if discrete, but consider the simplest continuous case first; 

Find e(l), e(2) such that ^(1) + ^(2) = 1, gfi = c 

Then, (5 is a Bayes decision rule. 

C(l)ai(5) +^(2)q2(5) < ^(l)ai(<5') +$(2)a2(<5') 

for any decision rule 5' 

If 5' e Ka then ai{S') < a. 

Note; ai{S) = a, so; ^{l)a + ^{2)a2{6) < ^{l)a + ^{2)a2{d') 

Therefore: a2{S) < a2{S'), 5 is the best (mosst powerful) decision rule in Ka 

Example: 

Hi : N{0,1),H2 ; iV(l, 1), ai(,5) = 0.05 
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/l(x) _1v-„2_l1 



/2(X) 

Always simplify first: 



2 

The decision rule becomes: 



> log(c),^a;i < ^ + log(c),^a;i < c' 



(5 = {Fi : < c';if2 : ^a;^ > c'} 

Now, find c' 

aiW = Pi(E^i>c') 

recall, subscript on P indicates that X\, a;„ ~ N{0, 1) 
Make into standard normal: 

Pi(^>^)=0.05 
Check the table for > c") = 0.05, c" = 1.64, c' = ^/n(1.64) 
Note: a very common error with the central limit theorem: 

These two conversions are the same! Don't combine techniques from both. 
The Bayes decision rule now becomes: 

5 = : < 1.64VE;H2 : > 1.64^^} 

Error of Type 2: 

a2((5) =P2(E^i < c = 1.64V^) 

Note: subscript indicates that Xi, ...,X„ N{1, 1) 

= F,P^^^ < ^■^^^-" ) = F,iz < 1.64 - v^) 

Use tables for standard normal to get the probability. 
If n = 9 ^ P2{z < 1.64 - V9) = F2{z < -1.355) = 0.0877 

Example: 

Hi : iV(0, 2), i?2 : iV(0, 3), ai((5) = 0.05 

/2(x) ( 1 w-^^-? ^2> ^ 

This is intuitive, as the sum of squares ~ sample variance. 
If small a = 2 
If large — > = 3 
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a^{6) = Pi(Ea:f > c') = Pi(^ ^ > ^) = P.ixl > c") 
If n = 10, Pi(xfo > c") = 0.05; c" = 18.31, c' = 36.62 

Can find error of type 2 in tlie same way as earlier: 

nxi >^)^ nxio > 12-1) « 0.7 

A difference of 1 in variance is a liuge deal! 
Large type 2 error results, small n. 

** End of Lecture 30 
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